Papers with computational consumption
Be More with Less: Hypergraph Attention Networks for Inductive Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Text classification is a critical research topic with broad applications in natural language processing. graph neural networks (GNNs) have received increasing attention but their performance is jeopardized in practice. |
| Approach: | They propose a model which captures long-distance interactions between words and a graph-based model which can be used to perform text classification. |
| Outcome: | The proposed model can achieve more expressive power with less computational consumption on the text classification task. |
DB-LLM: Accurate Dual-Binarization for Efficient LLMs (2024.findings-acl)
Copied to clipboard
Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, Dacheng Tao
| Challenge: | Existing methods for ultra-low bit quantization cause severe accuracy drops . a novel Dual-Binarization method is proposed for efficient Large Language Models . |
| Approach: | They propose a Dual-Binarization method that takes 2-bit-width and binarization into account . they propose DB-LLM, which uses a 2-bit binarized weighted model to represent weights efficiently . |
| Outcome: | The proposed method surpasses the current State-of-the-Art in ultra-low bit quantization and achieves 20% reduction in computational consumption compared to the SOTA method under the same bit-width. |